1. Field of the Invention
The invention relates to image processing methods for reducing noise in a sampled image. More specifically, the invention pertains to an image processing method which reduces noise while minimizing unintended distortion of image features.
2. Description Relative to the Prior Art
Pictures generated by an image processing method often display artifacts introduced by the processing method itself. Such artifacts may mask the benefits obtained by processing the image. This invention pertains to the suppression of a particular class of artifacts: the introduction of what appear to be "edges" into places--like smooth facial features--where no such edges existed in the original image. To better describe the invention it is necessary to review certain aspects of known image processing technology.
In known image processing methods, every image signal is replaced by a modified value, based on the values of the image signals from a surrounding field of image elements. The signals from the surrounding field are used to form a number of different linear combinations each of which represents a different component of the image structure within the field. In a typical method, most of the combinations represent the detail within the field. Each detail-sensitive combination represents a difference among local image signals and tends to vanish in the absence of a particular kind of image detail. Noise is reduced by modifying the detailsensitive combinations such that, for example, the value of a combination is lowered or set to zero wherever a particular kind of image detail is not present.
One such method, hereinafter referred to as a point-neighborhood method, considers the signal representative of the light value of each original image element in combination with similar signals from a small neighborhood of surrounding elements. (Light value, as used herein, shall mean any imagerelated characteristic--e.g., lightness, brightness, image density, hue and the like--that can be expressed in a form suitable for image processing.) The point-neighborhood method reconstitutes the image by replacing the signal from each original image element with the sum of a number of linear combinations of the neighborhood signals. The general plan of such a method is to 1) generate image signals representative of light values by sampling the image according to a regular array of picture elements, 2) filter the image signals to obtain a low-pass and one or more band-pass image signals (one of which may be high-pass) 3) core each band-pass signal by comparison to a threshold to remove noise, 4) amplify the cored band-pass signals to increase sharpness, and 5) add the amplified, cored band-pass signals to the low-pass signal to obtain a final image for display or reproduction. (Coring is a non-linear noise reduction process that removes signal energy--presumably noise--near the average signal axis and less than a threshold; the remaining signal is then added back to the low-pass signal. See "Digital Techniques of Reducing Television Noise," by J. P. Rossi, Journal of the Society of Motion Picture and Television Engineers, March 1978, pp. 134-140. A complementary process, hereinafter referred to as clipping, removes signal energy--presumably image detail--that is above a threshold; the remaining noise signal is then subtracted from the full-band signal.)
One version of a point-neighborhood method divides the image into (A) a low-pass image signal obtained by convolving the original image with 1/16 of a 3 by 3 array of weights such as
______________________________________ 1 2 1 (1) 2 4 2 1 2 1 ______________________________________
and (B) a high-pass image signal composed of eight high-pass component signals obtained by convolving the original image with 1/16 of a combination of eight 3 by 3 arrays of weights such as
______________________________________ 0 00 -1 00 0 -2 0 0 0 -1 (2) -2 20 0 10 02 0 0 10 0 00 0 00 00 0 0 00 0 00 0 00 00 0 0 00 0 2 -2 0 10 02 0 0 10 0 00 0 0 -1 0 -2 0 -1 00 ______________________________________
Each high-pass image component signal is proportional to the image gradient in a particular direction. A sharpened image of reduced noise is obtained by coring each high-pass component signal, amplifying each cored high-pass component signal and adding the amplified component signals to the low-pass signal. Such point-neighborhood processing methods are described in (A) commonly assigned copending patent application Ser. No. 357,357, entitled "Electronic Image Processing," filed Mar. 12, 1982 as a continuation of patent application Ser. No. 192,954, filed May 28, 1980 and (B) "A Method for the Digital Enhancement of Unsharp, Grainy Photographic Images," by P. G. Powell and B. E. Bayer, Proceedings of the International Conference on Electronic Image Processing, July 26-28, 1982, pp. 179-183.
The known method thus far described removes noise of very high frequency and responds only to local aspects of an image gradient, whether in fact the overall gradient is local or extended. In order to process lower frequency noise in a point-neighborhood method, an even lower-pass signal is obtained in a second stage by convolving the low-pass signal of the initial stage heretofore described with 1/16 of a 5 by 5 array of weights such as
______________________________________ 1 0 2 0 1 (3) 0 0 0 0 0 2 0 4 0 2 0 0 0 0 0 1 0 2 0 1 ______________________________________
Just as eight high-pass, or fine detail, image component signals were obtained from the original image, eight band-pass, or intermediate detail, image component signals are now obtained from the low-pass signal by convolving that signal with 1/16 of a combination of eight 5 by 5 arrays of weights, such as
______________________________________ 0 0 0 00 -1 0 0 00 0 0 -2 0 0 0 0 0 0 -1 (4) 0 0 0 00 0 0 0 00 0 00 0 0 0 0 0 00 -2 0 2 00 0 0 1 00 0 02 0 0 0 0 1 00 0 0 0 00 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 0 0 0 00 0 00 0 0 0 0 0 00 0 0 2 0 -2 0 0 1 00 0 02 0 0 0 0 1 00 0 0 0 00 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 0 0 0 0 -1 0 0 -2 0 0 -1 0 0 00 ______________________________________
As before, each band-pass image component signal is proportional to the average gradient in a particular direction, but now the result is taken over a larger area. These image component signals are cored and summed with the lower-pass signal to produce a smoothed version of the original low-pass signal, which is then added to the processed high-pass image component signals from the initial, or first, stage (such cascading procedures are described in the previously cited Powell and Bayer article and in commonly assigned, copending patent applications Ser. Nos. 328,543 and 328,544, both entitled "Electronic Image Processing" and both filed Nov. 30, 1981.) This process can be cascaded any number of times until removing noise of lower frequency yields little further improvement.
In the above-described point-neighborhood method, noise is reduced by coring (or clipping) the detail-sensitive linear combinations of the image signals. Since the noise coring process involves the application of a non-linear function (e.g., a threshold), some distortion of local image values may be generated as an artifact of the processing itself, but this is tolerated in order to realize the desired noise reduction. For example, none of the gradient-sensitive weighting arrays heretofore discussed will generate component signals that respond to the abrupt change of a low contrast edge without also responding to a gradual change in a smooth, extended image gradient--such as is frequently found within smooth areas of scene objects. A local gradient (i.e., a low contrast edge) and an extended gradient may thus "look" the same to a weighting array that encompasses only the local field.
The non-linear coring procedure is in part justified by the assumption that transition between the cored and non-cored states is mostly acceptable in a "busy" region of the image, as at an edge. The problem arises where the local and extended gradients appear the same, that is, in certain less "busy" regions of an image where the light value is changing only smoothly and slowly. In such regions the value of one or more of the detail-sensitive linear combinations will pass through its noise threshold. Because this situation activates the coring procedure, an abrupt discontinuity will undesirably appear in the processed image at the point where the threshold is crossed and the corresponding linear combination is undesirably modified. In less "busy" regions--like the smoothed area of an extended gradient--this transition sometimes leads to a visible artifact--much like an "edge"--and therefore is undesirable. From an aesthetic viewpoint, such artifacts particularly detract from the overall visual appeal of images reproduced by such methods. In fact, in some areas of an image such transitions may be more objectionable than the original noise component that coring has removed. Point-neighborhood methods of which I am aware are unable to effectively deal with these types of artifacts, therefore yielding aesthetically unappealing results. My invention provides a solution for this type of problem.